Moments, and moment generating functions

The Cantor distribution

Let . Delete the middle third of and let ,be the remaining set, and continue similarly to obtain an infinite nested sequence of subsets of . The Cantor set is defined to be the limit There is another way of thinking about the Cantor set, and this is useful for us. Just as each number in has an expansion in the base-10 system (namely its decimal expansion) so it has an expansion in the base-3 system. That is to say, any may be written in the form The Cantor set is the set of all points for which the above take the values 0 and 2 only We define by Let be a random variable with distribution function , is neither discrete nor continuous.It turns out that the distribution function is in an entirely new category, called the set of ‘singular’ distribution functions.

• Decomposition Theorem:every distribution function may be expressed in the form for non-negative summing to , such that: is the distribution function of a discrete random variable, that of a continuous random variable, and is singular.

Moments

• For any random variable , the moment of is defined for to be the number , that is, the expectation of the power of , whenever this expectation exists.
• Theorem 7.7 (Uniqueness theorem for moments)Suppose that all moments of the random variable exist, and that the series is absolutely convergent for some . Then the sequence of moments uniquely determines the distribution of .

Variance and covariance

• if and only if so that ‘zero variance’ means ‘no dispersion at all’.
• As a measure of dispersion, the variance of has an undesirable property: it is non-linear in the sense that the variance of is times the variance of . For this reason, statisticians often prefer to work with the standard deviation of , defined to be.
• Definition 7.19 The covariance of the random variables and is the quantity denoted and given by whenever these expectations exist.
• ,
• If and are independent, then,,then ,The converse of this is false in general.
• is often used as a measure of the dependence of and , and the reason for this is that is a single number
• A principal disadvantage of covariance as a measure of dependence is that it is not ‘scale-invariant’
• Definition 7.25 The correlation (coefficient) of the random variables and is the quantity given by whenever the latter quantities exist and .so correlation is scale invariant
• Theorem 7.28 If and are random variables, then whenever this correlation exists
• Theorem 7.30 (Cauchy–Schwarz inequality) If and are random variables, then
• if and are independent, then ,we say that and are uncorrelated
• is a linear increasing function of if and only if ,
• is a linear decreasing function of if and only if .

Moment generating functions

• Definition 7.39 The moment generating function (or mgf) of the random variable is the function defined by for all for which this expectation exists.
• Theorem 7.49 If exists in a neighborhood of , then, for , . . . , the derivative of evaluated at .
• Consider first the linear function of the random variable . If ,
• Theorem 7.52 If and are independent random variables, then has moment generating function
• By Theorem 7.52, the sum of independent random variables has moment generating function
• Theorem 7.55 (Uniqueness theorem for moment generating functions) If the moment generating function satisfies for all t satisfying and some , there is a unique distribution with moment generating function . Furthermore, under this condition, we have that .and

Two inequalities

• Theorem 7.63 (Markov’s inequality) For any non-negative random variable ,
• A function g : (a, b) → R is called convex if
• Theorem 7.67 (Jensen’s inequality) Let be a random variable taking values in the (possibly infinite) interval such that exists, and let be a convex function such that . Then
• Theorem 7.68 (Supporting tangent theorem) Let be convex, and let . There exists such that

Characteristic functions

• Definition 7.76 The characteristic function of the random variable is defined to be the function given by where
• If the moment generating function is finite in a non-trivial neighborhood of the origin, the characteristic function of may be found by substituting in the formula for :
• Under this condition, it follows that the moments of may be obtained in terms of the derivatives of : the derivative of at 0.
• Theorem 7.85 If for some positive integer , then
• Theorem 7.87 Let and be independent random variables with characteristic functions and , respectively.
  • If and , then .
  • The characteristic function of is .
• Theorem 7.88 (Uniqueness theorem for characteristic functions) Let and have characteristic functions and , respectively. Then and have the same distributions if and only if for all .
• Theorem 7.89 (Inversion theorem) Let X have characteristic function φ and density function f . Then